Holoviews render tests
Contents
Holoviews render tests#
31/01/23
Working OK for wrapping Plotly surfaces (PAD plots, see various plotly test notebooks), but failing for Bokeh plots (2D matrix plots)…?
Ran into issue for density matrix plotting in theory chpt, see http://jake:9966/lab/tree/QM3/doc-source/part1/theory_observables_intro_211122.ipynb
Original code from MF recon manuscript, which pushed to static layout, see http://jake/jupyter/user/paul/doc/tree/code-share/stimpy-docker-local/MFPADs_recon_manuscript_dev_April_2022/MFrecon_manuscript_fig_generation_170422-Stimpy_MAIN-oldPkgs.ipynb. Note this has hvSave code, not sure if this made it elsewhere.
UPDATE 31/01/23 pm: worked on first build (for HTML output)… but not later? WTF? Actually, was OK eventually, just had to reload page and wait (issues with Ice Dragon rendering?). Still not working in main page however, maths bug issue?
Setup#
# Run default config - may need to set full path here
%run '../scripts/setup_notebook.py'
# Override plotters backend?
# plotBackend = 'pl'
*** Setting up notebook with standard Quantum Metrology Vol. 3 imports...
For more details see https://pemtk.readthedocs.io/en/latest/fitting/PEMtk_fitting_basic_demo_030621-full.html
To use local source code, pass the parent path to this script at run time, e.g. "setup_fit_demo ~/github"
Running: 2023-02-09 17:55:05
Working dir: /home/jovyan/jake-home/buildTmp/_latest_build/html/doc-source/tests
Build env: html
* Loading packages...
* sparse not found, sparse matrix forms not available.
* natsort not found, some sorting functions not available.
* Setting plotter defaults with epsproc.basicPlotters.setPlotters(). Run directly to modify, or change options in local env.
* Set Holoviews with bokeh.
* pyevtk not found, VTK export not available.
***xyzpy not found, parallel functions not available.
* Set Holoviews with bokeh.
Jupyter Book : 0.13.2
External ToC : 0.3.1
MyST-Parser : 0.15.2
MyST-NB : 0.13.2
Sphinx Book Theme : 0.3.3
Jupyter-Cache : 0.4.3
NbClient : 0.5.4
OMP: Info #271: omp_set_nested routine deprecated, please use omp_set_max_active_levels instead.
# Setup symmetry-defined matrix elements using PEMtk
# Import class
from pemtk.sym.symHarm import symHarm
# Compute hamronics for Td, lmax=4
sym = 'D2h'
lmax=4
lmaxPlot = 2 # Set lmaxPlot for subselection on plots later.
# Glue items for later
glue("symHarmPGmatE", sym, display=False)
glue("symHarmLmaxmatE", lmax, display=False)
glue("symHarmBasislmaxPlot", lmaxPlot, display=False)
# TODO: consider different labelling here, can set at init e.g. dims = ['C', 'h', 'muX', 'l', 'm'] - 25/11/22 code currently fails for mu mapping, remap below instead
symObj = symHarm(sym,lmax)
# symObj = symHarm(sym,lmax,dims = ['Cont', 'h', 'muX', 'l', 'm'])
# To plot using ePSproc/PEMtk class, these values can be converted to ePSproc BLM data type...
# Run conversion - the default is to set the coeffs to the 'BLM' data type
dimMap = {'C':'Cont','mu':'muX'}
symObj.toePSproc(dimMap=dimMap)
# Run conversion with a different dimMap & dataType
dataType = 'matE'
# symObj.toePSproc(dimMap = {'C':'Cont','h':'it', 'mu':'muX'}, dataType=dataType)
symObj.toePSproc(dimMap = dimMap, dataType=dataType)
# symObj.toePSproc(dimMap = {'C':'Cont','h':'it'}, dataType=dataType) # Drop mu > muX mapping for now
# symObj.coeffs[dataType]
# Example using data class (setup in init script)
data = pemtkFit()
# Set to new key in data class
dataKey = sym
data.data[dataKey] = {}
for dataType in ['matE','BLM']:
data.data[dataKey][dataType] = symObj.coeffs[dataType]['b (comp)'].sum(['h','muX']) # Select expansion in complex harmonics, and sum redundant dims
data.data[dataKey][dataType].attrs = symObj.coeffs[dataType].attrs
Remapped dims: {'C': 'Cont', 'mu': 'muX'}
Added dim Eke
Added dim P
Added dim T
Added dim C
Remapped dims: {'C': 'Cont', 'mu': 'muX'}
Added dim Eke
Added dim Targ
Added dim Total
Added dim mu
Added dim it
Added dim Type
# Compute basis functions for given matrix elements
# Set data
data.subKey = dataKey
# Using PEMtk - this only returns the product basis set as used for fitting
BetaNormX, basisProduct = data.afblmMatEfit(selDims={}, sqThres=False)
# Using ePSproc directly - this includes full basis return if specified
BetaNormX2, basisFull = ep.geomFunc.afblmXprod(data.data[data.subKey]['matE'], basisReturn = 'Full', selDims={}, sqThres=False) #, BLMRenorm = BLMRenorm, **kwargs)
# The basis dictionary contains various numerical parameters, these are investigated below.
# See also the ePSproc docs at https://epsproc.readthedocs.io/en/latest/methods/geometric_method_dev_260220_090420_tidy.html
print(f"Product basis elements: {basisProduct.keys()}")
print(f"Full basis elements: {basisFull.keys()}")
# Use full basis for following sections
basis = basisFull
Product basis elements: dict_keys(['BLMtableResort', 'polProd', 'phaseConvention', 'BLMRenorm'])
Full basis elements: dict_keys(['QNs', 'EPRX', 'lambdaTerm', 'BLMtable', 'BLMtableResort', 'AFterm', 'AKQS', 'polProd', 'phaseConvention', 'BLMRenorm'])
# DEMO CODE FROM http://jake/jupyter/user/paul/doc/tree/code-share/stimpy-docker-local/MFPADs_recon_manuscript_dev_April_2022/MFrecon_manuscript_fig_generation_170422-Stimpy_MAIN-oldPkgs.ipynb
# SEE ALSO DOCS, https://epsproc.readthedocs.io/en/dev/methods/density_mat_notes_demo_300821.html#Density-Matrices
# Import routines
from epsproc.calc import density
# Compose density matrix
# Set dimensions/state vector/representation
# These must be in original data, but will be restacked as necessary to define the effective basis space.
denDims = 'LM' #, 'mu']
selDims = None #{'Type':'L'}
pTypes=['r','i']
thres = 1e-4 # 0.2 # Threshold out l>3 terms if using full 'orb5' set.
normME = False
normDen = 'max'
# Calculate - Ref case
# matE = data.data['subset']['matE']
# Set data from master class
# k = 'orb5' # N2 orb5 (SG) dataset
# k = 'subset'
k = sym
matE = data.data[k]['matE']
if normME:
matE = matE/matE.max()
daOut, *_ = density.densityCalc(matE, denDims = denDims, selDims = selDims, thres = thres) # OK
if normDen=='max':
daOut = daOut/daOut.max()
elif normDen=='trace':
daOut = daOut/(daOut.sum('Sym').pipe(np.trace)**2) # Need sym sum here to get 2D trace
# daPlot = density.matPlot(daOut.sum('Sym'))
daPlot = density.matPlot(daOut.sum('Sym'), pTypes=pTypes)
# # Retrieved
# matE = data.data['agg']['matE']['compC']
# if normME:
# matE = matE/matE.max()
# daOut2, *_ = density.densityCalc(matE, denDims = denDims, selDims = selDims, thres = thres) # OK
# if normDen=='max':
# daOut2 = daOut2/daOut2.max()
# elif normDen=='trace':
# daOut2 = daOut2/(daOut2.sum('Sym').pipe(np.trace)**2)
# daPlot2 = density.matPlot(daOut2.sum('Sym'), pTypes=pTypes) #.sel(Eke=slice(0.5,1.5,1)))
# # Compute difference
# daDiff = daOut.sum('Sym') - daOut2.sum('Sym')
# daDiff.name = 'Difference'
# daPlotDiff = density.matPlot(daDiff, pTypes=pTypes)
# #******** Plot
# daLayout = (daPlot.layout('pType') + daPlot2.opts(show_title=False).layout('pType').opts(show_title=False) + daPlotDiff.opts(show_title=False).layout('pType')).cols(1) # No cols? AH - set to 1 works.
# # daLayout.opts(width=300, height=300) # Doesn't work?
# daLayout.opts(hvPlotters.opts.HeatMap(width=300, frame_width=300, aspect='square', tools=['hover'], colorbar=True, cmap='coolwarm')) # .opts(show_title=False) # .opts(title="Custom Title") #OK
Set plot kdims to ['LM', 'LM_p']; pass kdims = [dim1,dim2] for more control.
# OPTIONAL PLOT SETTINGS?
# General size & range unless overridden
figSize = [700,300]
tRange = [3.8, 5.2] # Set for axis slice, also for cmapping lims
# Update defaults
ep.plot.hvPlotters.setPlotters(width = figSize[0], height = figSize[1], snsStyle = 'white')
* Set Holoviews with bokeh.
# Raw plot
daPlot
WARNING:param.HeatMapPlot04089: Due to internal constraints, when aspect and width/height is set, the bokeh backend uses those values as frame_width/frame_height instead. This ensures the aspect is respected, but means that the plot might be slightly larger than anticipated. Set the frame_width/frame_height explicitly to suppress this warning.
#******** Plot with layout
from epsproc.plot import hvPlotters # Additional plotting code
daLayout = (daPlot.layout('pType')).cols(1) # No cols? AH - set to 1 works.
# daLayout.opts(width=300, height=300) # Doesn't work?
daLayout.opts(hvPlotters.opts.HeatMap(width=300, frame_width=300, aspect='square', tools=['hover'], colorbar=True, cmap='coolwarm')) # .opts(show_title=False) # .opts(title="Custom Title") #OK
WARNING:param.HeatMapPlot04194: Due to internal constraints, when aspect and width/height is set, the bokeh backend uses those values as frame_width/frame_height instead. This ensures the aspect is respected, but means that the plot might be slightly larger than anticipated. Set the frame_width/frame_height explicitly to suppress this warning.
WARNING:param.HeatMapPlot04202: Due to internal constraints, when aspect and width/height is set, the bokeh backend uses those values as frame_width/frame_height instead. This ensures the aspect is respected, but means that the plot might be slightly larger than anticipated. Set the frame_width/frame_height explicitly to suppress this warning.
# Test with glue
glue("glueTest", daPlot, display=False)
WARNING:param.HeatMapPlot04246: Due to internal constraints, when aspect and width/height is set, the bokeh backend uses those values as frame_width/frame_height instead. This ensures the aspect is respected, but means that the plot might be slightly larger than anticipated. Set the frame_width/frame_height explicitly to suppress this warning.
Fig. 1 Raw output to glue.#
Normal glue fails for PDF output (but OK for HTML).
Feb 2023: Now have wrapper glueHV() for this…
# Test with glueHV()
glueHV("glueTestGHV", daPlot)
glueHV("realOnlyGHV", daPlot.select(pType='Real'))
WARNING:param.HeatMapPlot04311: Due to internal constraints, when aspect and width/height is set, the bokeh backend uses those values as frame_width/frame_height instead. This ensures the aspect is respected, but means that the plot might be slightly larger than anticipated. Set the frame_width/frame_height explicitly to suppress this warning.
WARNING:param.HeatMapPlot04369: Due to internal constraints, when aspect and width/height is set, the bokeh backend uses those values as frame_width/frame_height instead. This ensures the aspect is respected, but means that the plot might be slightly larger than anticipated. Set the frame_width/frame_height explicitly to suppress this warning.
Fig. 2 Raw output to glueHV.#
Fig. 3 Real only output to glueHV.#
Test new glue wrapper/decorator (09/02/23)#
See https://realpython.com/primer-on-python-decorators/
09/02/23: basically working, but for Holomaps may have issues with stacked dims. Also seems to change/force cmap on save? TBC.
UPDATE: now implemented in setup_notebook.py too.
# def superglue(func):
def glueDecorator(func):
'''Decorator for glue() with interactive plot types forced to static for PDF builds.'''
def glueWrapper(name,fig,**kwargs):
# Set names for file out
# Note imgFormat and imgPath should be set globally, or passed as kwargs.
imgFile = f'{name}.{imgFormat}'
imgFile = os.path.join(imgPath,imgFile)
# Set glue() output according to fig type and build env.
# Note buildEnv should be set globally, or passed as a kwarg.
if buildEnv != 'pdf':
return func(name, fig, display=False) # For non-PDF builds, use regular glue()
else:
# Holoviews object
# NOTE this may give unexpected results in some cases for Holomaps - may want to force flatten?
# Note for Bokeh backend may need additional pkgs, selenium, firefox and geckodriver
# See https://holoviews.org/user_guide/Plots_and_Renderers.html#saving-and-rendering
if 'holoviews' in str(type(fig)):
# Force render and glue
hv.save(fig, imgFile, fmt=imgFormat)
elif 'plotly' in str(type(fig)):
fig.write_image(imgFile,format=imgFormat) # See https://plotly.com/python/static-image-export/
# Glue static render
return func(name, Image(imgFile), display=False)
return glueWrapper
glue = glueDecorator(glue)
glue("glueTestDec", daPlot)
WARNING:param.HeatMapPlot04398: Due to internal constraints, when aspect and width/height is set, the bokeh backend uses those values as frame_width/frame_height instead. This ensures the aspect is respected, but means that the plot might be slightly larger than anticipated. Set the frame_width/frame_height explicitly to suppress this warning.
Fig. 4 Raw output to glue decorated with superglue().#
daLayout = (daPlot.layout('pType')).cols(1)
glue("glueTestDecLayout", daLayout)
WARNING:param.HeatMapPlot04476: Due to internal constraints, when aspect and width/height is set, the bokeh backend uses those values as frame_width/frame_height instead. This ensures the aspect is respected, but means that the plot might be slightly larger than anticipated. Set the frame_width/frame_height explicitly to suppress this warning.
WARNING:param.HeatMapPlot04483: Due to internal constraints, when aspect and width/height is set, the bokeh backend uses those values as frame_width/frame_height instead. This ensures the aspect is respected, but means that the plot might be slightly larger than anticipated. Set the frame_width/frame_height explicitly to suppress this warning.
Fig. 5 Raw layout to glue decorated with superglue().#
type(daLayout)
holoviews.core.layout.NdLayout
Check Plotly compatibility#
For original tests see http://jake:9966/lab/tree/QM3/doc-source/tests/plotly_pdf_export_test_181122.ipynb
# Test from https://github.com/executablebooks/jupyter-book/issues/1410#issuecomment-984661412
import plotly.graph_objects as go
# import plotly.io as pio
# pio.renderers.default = "png" # This works for PDF export, but also forces png in HTML case.
ifig = go.Figure(go.Scatter(x=[1,2], y=[1,2]))
ifig.show()
type(ifig)
plotly.graph_objs._figure.Figure
glue("glueTestPlotly", ifig)
WARNING:param.Plotly04509: Setting non-parameter attribute display=False using a mechanism intended only for parameters
---------------------------------------------------------------------------
ValueError Traceback (most recent call last)
/tmp/ipykernel_37437/4129629514.py in <module>
----> 1 glue("glueTestPlotly", ifig)
/tmp/ipykernel_37437/3749546079.py in glueWrapper(name, fig, **kwargs)
13 # Note buildEnv should be set globally, or passed as a kwarg.
14 if buildEnv != 'pdf':
---> 15 return func(name, fig, display=False) # For non-PDF builds, use regular glue()
16
17 else:
~/jake-home/buildTmp/_latest_build/html/doc-source/scripts/setup_notebook.py in glueWrapper(name, fig, **kwargs)
279 # Without Panel some basic plot types work, but not surface plots - may also be browser-dependent?
280 # Or due to maths bug, per https://jupyterbook.org/en/stable/interactive/interactive.html#plotly
--> 281 return glue(name, pn.pane.Plotly(fig, **kwargs), display=False)
282
283 else:
~/jake-home/buildTmp/_latest_build/html/doc-source/scripts/setup_notebook.py in glueWrapper(name, fig, **kwargs)
279 # Without Panel some basic plot types work, but not surface plots - may also be browser-dependent?
280 # Or due to maths bug, per https://jupyterbook.org/en/stable/interactive/interactive.html#plotly
--> 281 return glue(name, pn.pane.Plotly(fig, **kwargs), display=False)
282
283 else:
/opt/conda/lib/python3.9/site-packages/panel/pane/plotly.py in __init__(self, object, **params)
95
96 def __init__(self, object=None, **params):
---> 97 super().__init__(object, **params)
98 self._figure = None
99 self._event = None
/opt/conda/lib/python3.9/site-packages/panel/pane/base.py in __init__(self, object, **params)
140 applies = self.applies(object, **(params if self._applies_kw else {}))
141 if (isinstance(applies, bool) and not applies) and object is not None :
--> 142 self._type_error(object)
143
144 super().__init__(object=object, **params)
/opt/conda/lib/python3.9/site-packages/panel/pane/base.py in _type_error(self, object)
158
159 def _type_error(self, object):
--> 160 raise ValueError("%s pane does not support objects of type '%s'." %
161 (type(self).__name__, type(object).__name__))
162
ValueError: Plotly pane does not support objects of type 'Plotly'.
Fig. 6 Plotly figure to glue decorated with superglue().#